1. Field of the Invention
The present invention pertains to the field of digital video processing and relates to a video coding device for efficiently encoding video data and a video decoding device for decoding video data coded by the video coding device.
2. Description of Related Art
Recently, there has been proposed a subband coding method that can efficiently encode and decode video signals. The well-known high-efficient subband encoding method is used to decompose an input image into frequency bands by a bank of band-decomposing filters. The band-decomposing filter-bank is a one-dimensional filter-bank that can serve as a two-dimensional band-decomposing filter-bank by repeating processing the input image in horizontal and vertical directions. This method was reported by Fujii, Noumura. xe2x80x9cTopics on Wavelet Transformxe2x80x9d in a Report of xe2x80x9cTECHNICAL REPORT of IEICE, IE92-11, 1992xe2x80x9d.
In the prior art, a subband image as shown in FIG. 1B obtained by conducting two-dimensional subband decomposition three times. The first two-dimensional subband decomposition obtains a horizontal high-pass and a vertical low-pass band, a horizontal low-pass and vertical high-pass band and a horizontal and vertical high-pass band, which are designated by HL1, LH1 and HH1 respectively. A horizontal and vertical low-pass band obtained by the first decomposition is further subjected to two-dimensional band-decomposition by which three subbands HL2, LH2 and HH2 are obtained.
A horizontal and vertical low-pass subband obtained by the second decomposition is further subjected to third two-dimensional subband decomposition by which three subbands HL3, LH3 and HH3 and a horizontal and vertical low-pass subband LL3 are obtained. A Wavelet-converting filter-bank or a band-decomposing and synthesizing filter-bank may be used as the band-decomposing filter-bank. Thus, the decomposed subband-images are of a hierarchical (layer) structure from low-frequency band to high-frequency band.
Progressive image transmitting can be easily realized utilizing the hierarchical structure of the subband images. The progressive image transmitting method enables a video decoding device to reproduce a low-resolution image by using only a part of coded data. The more coded data is reproduced, the higher resolution the decoded image has. Japanese Laid-Open Patent Publication (TOKKAI HEI) No. 8-242379 describes a system (referred hereinafter to as a prior art system) to realize the progressive image transmitting.
A video coding device using the prior art system includes a subband decomposing portion for decomposing an input image into subband images by using two-dimensional decomposing filters, a coefficient coding portion for encoding coefficients of the decomposed subband images, a variable-length coding portion for performing variable-length coding of the coded coefficient data from the coefficient coding portion and a line-transmitting portion for transmitting a plurality of components composing the image per line at a time. The coefficient coding portion performs encoding the coefficients by using any one of various kinds of coding methods (e.g., DPCM coding, zero-tree coding, and scalar-quantizing coding). This process includes a quantizing step.
The operation of the line transmitting portion will be described below in detail, by way of example, with an input image composed of three-components Y (a luminance component) and U, V (chrominance components) and being conducted subband decomposition three times as shown in FIG. 1B. Processing starts from a subband LL3, which gives the lowest resolution of the image.
In the example, the line-transmitting portion transmits the components Y, U and V sequentially line by line in the order from the first line of the subband LL3. Having transferred all lines of the subband LL3, the portion transfers the components Y, U and V in the subbands LH3, HL3 and HH3 respectively in the order: the component Y on the first lines of the subbands LH3, HL3 and HH3; the component U on the first lines of the subbands LH3, HL3 and HH3; the component V on the first lines of the subbands LH3, HL3 and HH3; the component Y on the second lines of the subbands LH3, HL3 and HH3; U on the second lines of LH3, HL3 and HH3; V on the second lines of the subbands LH3, HL3 and HH3 and so on. Having transmitted all lines of LH3, HL3 and HH3, the line transmitting portion transfers, in similar way, lines of LH2, HL2; HH2 and, then, lines of LH1, HL1, HH1. The above-mentioned procedure of the line-transmitting portion is executed according to a programed flow.
Orderly transmission of the components Y, U, V composing the image per line produces coded data having a hierarchical structure.
The prior art video decoding device includes a line receiving portion for receiving the coded data from the line-transmitting portion of the video-coding device above-mentioned and rearranging the data to respective component groups, a variable-length decoding portion for decoding the rearranged variable-length-coded data, a decoded data counting portion for counting bits of data decoded by the variable-length decoding portion, a decoding truncating portion for comparing the number of the bits counted by the decoded-data counting portion with a preset threshold or an externally-given threshold to give a command for stopping the decoding operation of the variable-length decoding portion when the number of decoded bits exceeds the threshold, a data completing portion for compensating for lack of truncated data by adding zero when having truncated the decoding the coded data at the specified number of bits, a coefficient decoding portion for decoding coded coefficient data by reversing the same processing procedure of the coefficient coding portion and a subband synthesizing portion for synthesizing an image from the subbands through two-dimensional synthesizing filters.
The video decoding device can thus reproduce an entire image from coded data having a hierarchical structure or a part thereof.
The conventional video-coding and video-decoding system can realize progressive image transmitting by transmitting image components per line in an ascending order starting from the lowest-resolution band-image. However, the prior art system encounters several inconvenient problems resulting from the fixed transfer-unit of a line. For example, an image composed of luminance component Y and chrominance components U and V may be easier recognized by transmitting only the component Y before the components U and V rather than transmitting all components as a unit.
In this case, it is preferable to transfer the image components subband by subband, not by line. Furthermore, it is proved that an image composed of components R, G, B may be reproduced with better subjective image-quality at the decoding terminal when coded coefficients of the respective components R, G and B are transmitted one by one. This is because these components have substantially the same influence on the visual property.
The prior art system presumes that components of an image have the same size. Therefore, it cannot be adaptable to an input image composed of different sizes of components in format of, e.g., 4:2:2 or 4:2:0.
Furthermore, the prior art system presumes that respective components of an image have the same number of subbands and cannot be adaptable to an input image whose components are divided into different numbers of subbands.
The present invention is directed to a system for effective progressive image transmitting by solving the foregoing problems involved in the prior arts.
(1) Accordingly, an object of the present invention is to provide a video coding device, which is provided with a subband-decomposing means for decomposing an image being composed of N (Nxe2x89xa72) kinds of luminance or chrominance components into subband images for each of components An (1xe2x89xa6nxe2x89xa6N, where n is an integer) composing an image to be coded, coefficient coding means for encoding a frequency coefficient of the subband images, rearranging means for preparing integrated component units by combining frequency coefficients included in respective components An according to the coded coefficient data prepared by the coefficient coding means and rearranging the prepared integrated component units of the coefficient-coded data in an ascending order of subband image resolution, starting from the integrated component unit including the coded coefficient data of the lowest resolution subband, and a variable-length coding means for performing variable-length encoding of the rearranged coefficient-coded data, wherein the rearranging means prepares each of the integrated component units by setting therein the frequency coefficients contained in the respective components An, which are all frequency-coefficients included in m (mxe2x89xa71) pieces of the respective components"" subbands, when the components An are have the same size and the same number of subbands.
(2) Correspondingly, another object of the present invention is to provide a video decoding device, which is provided with a variable-length decoding means for decoding variable-length coded data, a decoded-data counting means for counting bits of each integrated component unit decoded by the variable-length decoding means, a decoding truncating means for comparing the number of bits counted by the decoded-data counting means with an externally-given number of bits and giving a decoding-stop command when the number of decoded bits exceeds the given number of bits, a component separating means for separating the decoded integrated component unit into respective components An, a data completing means for compensating for lack of truncated data by adding a specified value to each of the components composing a screenful image, data arranging means for arranging coded coefficient data separated by the component separating means into specified positions for respective components An, a coefficient decoding means for decoding coded-coefficient data separated and arranged for respective components An by the component separating means, and a subband synthesizing means for reproducing a decoded image by combining subbands of data decoded by the coefficient decoding means for respective components An, wherein the component separating means separates the integrated component unit as combinations of all frequency coefficients contained in m (mxe2x89xa71) subbands for respective components An when the respective components An have the same size and the same number of subbands.
The integrated component units contains all frequency coefficients in m (mxe2x89xa71) respective subbands of respective component An. Therefore, specified subbands of the image components such as luminance signal Y and chrominance signals U and V that may have different levels of influence on human visual property can be transmitted first to enable one to recognize a summary of the image at an earlier stage of decoding at the decoding side. When a codable image is known to be of higher resolution in a specified direction, the coding device can transmit first coded coefficients of higher-resolution-direction subbands and the decoding device can decode those coded coefficients, terminate the decoding in the midway of decoding all coded data and reproduce the image from only data decoded till that time to improve subjective-image quality of the image.
(3) Another object of the present invention is to provide a video coding device, which is provided with a subband-decomposing means for decomposing an image being composed of N (Nxe2x89xa72) kinds of luminance or chrominance components into subband images for each of components An (1xe2x89xa6nxe2x89xa6N, where n is an integer) composing an image to be coded, coefficient coding means for encoding a frequency coefficient of the subband images, rearranging means for preparing integrated component units by combining frequency coefficients included in respective components An according to the coded coefficient data prepared by the coefficient coding means and rearranging the prepared integrated component units of the coefficient-coded data in an ascending order of subband image resolution, starting from the integrated component unit including the coded coefficient data of the lowest resolution subband, and a variable-length coding means for performing variable length encoding of the rearranged coefficient-coded data, wherein the rearranging means prepares each of the integrated component units by setting therein the frequency coefficients included in the respective components An as m (mxe2x89xa71) pieces of frequency-coefficients contained at the same relative positions in m (mxe2x89xa71) pieces of the respective components"" subbands when the components An have the same size and the same number of subbands.
(4) Correspondingly, another object of the present invention is to provide a video decoding device, which is provided with a variable-length decoding means for decoding variable-length coded data, a decoded-data counting means for counting bits of each integrated component unit decoded by the variable-length decoding means, a decoding truncating means for comparing the number of bits counted by the decoded-data counting means with an externally-given number of bits and giving a decoding-stop command when the number of decoded bits exceeds the given number of bits, a component separating means for separating the decoded integrated component unit into respective components An, a data completing means for compensating for lack of truncated data by adding a specified value to each of the components composing a screenful image, data arranging means for arranging coded coefficient data separated by the component separating means into specified positions for respective components An, a coefficient decoding means for decoding coded-coefficient data separated and arranged for respective components An by the component separating means, and a subband synthesizing means for reproducing a decoded image by combining subbands of data decoded by the coefficient decoding means for respective components An, wherein the component separating means separates the integrated component unit into combinations of m (mxe2x89xa71) pieces of frequency coefficients having the same relative positions in respective m (mxe2x89xa71) subbands of the respective components An when the components An have the same size and the same number of subbands.
Therefore, the devices operate with integrated component units whose elements are m (mxe2x89xa71) pieces of frequency coefficients having the same relative positions in m (mxe2x89xa71) respective subbands of respective components An and can decode those coded coefficients, terminate the decoding in the midway of decoding all the coded data and reproduce the image from only the data decoded till that time to improve subjective-image quality of the image when the image is composed of components R, G and B that have substantially almost the same influence on human visual property.
(5) Another object of the present invention is to provide a video coding device, which is provided with a subband-decomposing means for decomposing an image being composed of N (Nxe2x89xa72) kinds of luminance or chrominance components into subband images for each of components An (1xe2x89xa6nxe2x89xa6N, where n is an integer) composing an image to be coded, coefficient coding means for encoding a frequency coefficient of the subband images, rearranging means for preparing integrated component units by combining frequency coefficients included in respective components An according to the coded coefficient data prepared by the coefficient coding means and rearranging the prepared integrated component units of the coefficient-coded data in an ascending order of subband image resolution, starting from the integrated component unit including the coded coefficient data of the lowest resolution subband, and a variable-length coding means for performing variable-length encoding of the rearranged coefficient-coded data, wherein the rearranging means prepares each of the integrated component units by setting therein the different number of frequency-coefficients in the respective components An according to each component size when the components An are different in size and have the same number of subbands.
(6) Correspondingly, another object of the present invention is to provide a video decoding device, which is provided with a variable-length decoding means for decoding variable-length coded data, a decoded-data counting means for counting bits of each integrated component unit decoded by the variable-length decoding means, a decoding truncating means for comparing the number of bits counted by the decoded-data counting means with an externally-given number of bits and giving a decoding-stop command when the number of decoded bits exceeds the given number of bits, a component separating means for separating the decoded integrated component unit into respective components An, a data completing means for compensating for lack of truncated data by adding a specified value to each of the components composing a screenful image, data arranging means for arranging coded coefficient data separated by the component separating means into specified positions for respective components An, a coefficient decoding means for decoding coded-coefficient data separated and arranged for respective components An by the component separating means, and a subband synthesizing means for reproducing a decoded image by combining subbands of data decoded by the coefficient decoding means for respective components An, wherein the component separating means separates the integrated component unit as combinations of different pieces of frequency coefficients according to respective component sizes when the respective components An are different in size and have the same number of subbands.
The devices can be adapted to process an image whose luminance and chrominance components are different from each other by resolution, having the great advantage over the conventional method that can be applied to an image whose components have the same resolution. This feature provided by the present invention is desirable in particular to digital image processing since many digital images are usually formatted to have higher resolution of the luminance component than that of chrominance component.
(7) Another object of the present invention is to provide a video coding device, which is provided with a subband-decomposing means for decomposing an image being composed of N (Nxe2x89xa72) kinds of luminance or chrominance components into subband images for each of components An (1xe2x89xa6nxe2x89xa6N, where n is an integer) composing an image to be coded, coefficient coding means for encoding a frequency coefficient of the subband images, rearranging means for preparing integrated component units by combining the subbands included in respective components An according to the coded coefficient data prepared by the coefficient coding means and rearranging the prepared integrated component units of the coefficient-coded data in an ascending order of subband image resolution, starting from the integrated component unit including the coded coefficient data of the lowest resolution subband, and a variable-length coding means for performing variable-length encoding of the rearranged coefficient-coded data, wherein the rearranging means prepares each of the integrated component units by combining the same number of low-resolution subbands and the different number of high-resolution subbands of the respective components An when the components An are different in size and different in the number of subbands.
(8) Correspondingly, another object of the present invention is to provide a video decoding device, which is provided with a variable-length decoding means for decoding variable-length coded data, a decoded-data counting means for counting bits of each integrated component unit decoded by the variable-length decoding means, a decoding truncating means for comparing the number of bits counted by the decoded-data counting means with an externally-given number of bits and giving a decoding-stop command when the number of decoded bits exceeds the given number of bits, a component separating means for separating the decoded integrated component unit into respective components An, a data completing means for compensating for lack of truncated data by adding a specified value to each of the components composing a screenful image, data arranging means for arranging coded coefficient data separated by the component separating means into specified positions for respective components An, a coefficient decoding means for decoding coded-coefficient data separated and arranged for respective components An by the component separating means, and a subband synthesizing means for reproducing a decoded image by combining subbands of data decoded by the coefficient decoding means for respective components An, wherein the component separating means separates the integrated component unit as combinations of the same number of low-resolution subbands and the different number of high-resolution subbands of respective components An when the respective components An are different in size and different in the number of subbands.
The devices can be adapted to process an image whose luminance and chrominance components are different from each other by resolution and have different subband-decomposition levels, getting a great advantage over the conventional method that can be applied to an image whose components have the same resolution and the same number of subbands. This feature provided by the present invention is desirable in particular to digital image processing since many digital images are usually formatted to have higher resolution of the luminance component than that of the chrominance component and it is general to vary the subband-decomposition level according to the resolution of the component.
(9) Another object of the present invention is to provide a video coding device, which is provided with a subband-decomposing means for decomposing an image being composed of N (Nxe2x89xa72) kinds of luminance or chrominance components into subband images for each of components An (1xe2x89xa6nxe2x89xa6N, where n is an integer) composing an image to be coded, coefficient coding means for encoding a frequency coefficient of the subband images, rearranging means for preparing integrated component units by combining the subbands included in respective components An according to the coded coefficient data prepared by the coefficient coding means and rearranging the prepared integrated component units of the coefficient-coded data in an ascending order of subband image resolution, starting from the integrated component unit including the coded coefficient data of the lowest resolution subband, and a variable-length coding means for performing variable-length encoding of the rearranged coefficient-coded data, wherein the rearranging means prepares each of the integrated component units by combining the same number of high-frequency subbands and the different number of low-frequency subbands of the respective components An when the components An are different in size and different in the number of subbands.
(10) Correspondingly, another object of the present invention is to provide a video decoding device, which is provided with a variable-length decoding means for decoding variable-length coded data, a decoded-data counting means for counting bits of each integrated component unit decoded by the variable-length decoding means, a decoding truncating means for comparing the number of bits counted by the decoded-data counting means with an externally-given number of bits and giving a decoding-stop command when the number of decoded bits exceeds the given number of bits, a component separating means for separating the decoded integrated component unit into respective components An, a data completing means for compensating for lack of truncated data by adding a specified value to each of the components composing a screenful image, data arranging means for arranging coded coefficient data separated by the component separating means into specified positions for respective components An, a coefficient decoding means for decoding coded-coefficient data separated and arranged for respective components An by the component separating means, and a subband synthesizing means for reproducing a decoded image by combining subbands of data decoded by the coefficient decoding means for respective components An, wherein the component separating means separates the integrated component unit as combinations of the same number of high-resolution subbands and the different number of low-resolution subbands of respective components An when the components An are different sizes and different in the number of subbands.
The devices can be adapted to process an image whose luminance and chrominance components have different resolution levels and different subband-decomposition levels, getting a great advantage over the conventional method that can be applied to an image whose components have the same resolution and the same number of subbands. Furthermore, these aspects of the present invention provide such a feature that each integrated component unit always reflects the ratio of numbers of respective components contained in an input image. This feature eliminates the need for decoding redundant data at the decoding side when decoding the amount of data according to the resolution of the display unit.
(11) Another object of the present invention is to provide a video coding device, which is based on the device of (9) above-mentioned and further characterized in that the rearranging means prepares each of the integrated component units by combining lowest ones of resolution subbands of the respective components An and different numbers of all other low-resolution-level subbands of the respective components An when the respective components An are different in size and different in the number of subbands.
(12) Correspondingly, the present invention also provides a video-decoding device, which is based on the device of (10) above-mentioned and further characterized in that the component separating means separates the integrated component unit into combinations of subbands for respective components, each combination composed of one lowest resolution subband and the different numbers of all other low-resolution subbands.
The devices can first separate and transmit lowest resolution subbands of respective components An to first give a summary content of an image, making it possible to improve subjective quality of the reproduced image.